Sound processing apparatus and method, and program

ABSTRACT

A virtual speaker is assumed to exist on the lower side among the sides of a tetragon having its corners formed with four speakers surrounding a target sound image position on a spherical plane. Three-dimensional VBAP is performed with respect to the virtual speaker and the two speakers located at the upper right and the upper left, to calculate gains of the two speakers at the upper right and the upper left and the virtual speaker, the gains being to be used for fixing a sound image at the target sound image position. Further, two-dimensional VBAP is performed with respect to the lower right and lower left speakers, to calculate gains of the lower right and lower left speakers, the gains being to be used for fixing a sound image at the position of the virtual speaker. The values obtained by multiplying these gains by the gain of the virtual speaker are set as the gains of the lower right and lower left speakers for fixing a sound image at the target sound image position. The present technology can be applied to sound processing apparatuses.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit under 35U.S.C. § 120 of U.S. patent application Ser. No. 15/591,471, titled“SOUND PROCESSING APPARATUS AND METHOD, AND PROGRAM,” filed on May 10,2017, now U.S. Pat. No. 10,225,677, which is a continuation of andclaims the benefit under 35 U.S.C. § 120 of U.S. patent application Ser.No. 14/785,497, titled “SOUND PROCESSING APPARATUS AND METHOD, ANDPROGRAM,” filed on Oct. 19, 2015, now U.S. Pat. No. 9,681,249, which isthe National Stage of International Application No. PCT/JP2014/060459,filed in the Japanese Patent Office as a Receiving Office on Apr. 11,2014, which claims the priority benefit of Japanese Patent ApplicationNumber 2013-094268, filed in the Japanese Patent Office on Apr. 26,2013. Each of these applications is hereby incorporated by reference inits entirety.

TECHNICAL FIELD

The present technology relates to sound processing apparatuses andmethods, and programs, and more particularly, to a sound processingapparatus and method, and a program for enabling more stablelocalization of a sound image.

BACKGROUND ART

As a technique for controlling localization of a sound image usingspeakers, VBAP (Vector Base Amplitude Panning) has been known (seeNon-Patent Document 1, for example).

By VBAP, a target localization position of a sound image is expressed bya linear sum of vectors extending toward two or three speakers locatedaround the localization position. The coefficients by which therespective vectors are multiplied in the linear sum are used as thegains of the sounds to be output from the respective speakers, and gainadjustment is performed so that a sound image is fixed at the targetposition.

CITATION LIST Non-Patent Document

Non-Patent Document 1: Ville Pulkki, “Virtual Sound Source PositioningUsing Vector Base Amplitude Panning”, Journal of AES, vol. 45, no. 6,pp. 456-466, 1997

SUMMARY OF THE INVENTION Problems to be Solved by the Invention

By the above described technique, however, a sound image can be fixed ata target position, but localization of the sound image might becomeunstable depending on the localization position of the sound image.

By three-dimensional VBAP for performing VBAP using three speakers,there are cases where only two speakers among the three speakers outputsound, and the remaining one speaker is controlled not to output sounddepending on the target localization position of a sound image.

In such a case, when the user moves while listening to sound, the soundimage might move in a different direction from the direction of themovement, and the user might feel that localization of the sound imageis unstable. If localization of the sound image becomes unstable, therange of the sweet spot as an optimum listening position becomesnarrower.

The present technology has been developed in view of thosecircumstances, and is to enable more stable localization of a soundimage.

Solutions to Problems

A sound processing apparatus according to one aspect of the presenttechnology includes: a gain calculating unit that determines outputgains of sounds to be output from four or more sound outputting unitslocated close to a sound image localization position as a targetposition by calculating gains of sounds to be output from the soundoutputting units based on a positional relationship between the soundoutputting units with respect to each of different combinations amongcombinations of two or three of the four or more sound outputting units,the output gains being to be used for fixing a sound image at the soundimage localization position; and a gain adjusting unit that performsgain adjustment on sounds to be output from the sound outputting unitsbased on the output gains.

At least four of the output gains each have a value other than 0.

The gain calculating unit may include: a first gain calculating unitthat calculates output gains of a virtual sound outputting unit and twoof the sound outputting units based on a positional relationship amongthe virtual sound outputting unit, the two of the sound outputtingunits, and the sound image localization position; a second gaincalculating unit that calculates gains of other two of the soundoutputting units than the two of the sound outputting units based on apositional relationship among the other two of the sound outputtingunits and the virtual sound outputting unit, the gains of the other twoof the sound outputting units being to be used for fixing a sound imageat a position of the virtual sound outputting unit; and a calculatingunit that calculates output gains of the other two of the soundoutputting units based on the gains of the other two of the soundoutputting units and the output gain of the virtual sound outputtingunit.

The calculating unit may calculate the output gains of the other two ofthe sound outputting units by multiplying the gains of the other two ofthe sound outputting units by the output gain of the virtual soundoutputting unit.

The position of the virtual sound outputting unit may be set on a sideof a polygon having the four or more sound outputting units at thecorners thereof.

The gain calculating unit may include: a virtual gain calculating unitthat calculates output gains of three of the sound outputting unitsbased on a positional relationship among the three of the soundoutputting units and the sound image localization position; and acalculating unit that calculates ultimate output gains of the soundoutputting units based on the output gains calculated by the virtualgain calculating unit calculating the output gains with respect todifferent combinations among the combinations.

The calculating unit may calculate an ultimate output gain of the sameone of the sound outputting units by determining a sum of the outputgains determined with respect to the same one of the sound outputtingunits.

A sound processing method or program according to one aspect of thepresent technology includes the steps of: determining output gains ofsounds to be output from at four or more sound outputting units locatedclose to a sound image localization position as a target position bycalculating gains of sounds to be output from the sound outputting unitsbased on a positional relationship between the sound outputting unitswith respect to each of different combinations among combinations of twoor three of the four or more sound outputting units, the output gainsbeing to be used for fixing a sound image at the sound imagelocalization position; and performing gain adjustment on sounds to beoutput from the sound outputting units based on the output gains.

In one aspect of the present technology, output gains of sounds to beoutput from at four or more sound outputting units located close to asound image localization position as a target position are determined bycalculating gains of sounds to be output from the sound outputting unitsbased on a positional relationship between the sound outputting unitswith respect to each of different combinations among combinations of twoor three of the four or more sound outputting units, the output gainsbeing to be used for fixing a sound image at the sound imagelocalization position; and gain adjustment is performed on sounds to beoutput from the sound outputting units based on the output gains.

Effects of the Invention

According to one aspect of the present technology, a sound image can befixed in a more stable manner.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram for explaining two-dimensional VBAP.

FIG. 2 is a diagram for explaining three-dimensional VBAP.

FIG. 3 is a diagram for explaining speaker arrangement.

FIG. 4 is a diagram for explaining a gain calculation method to be usedin a case where four speakers are provided.

FIG. 5 is a diagram for explaining movement of a sound image.

FIG. 6 is a diagram for explaining movement of a sound image in a casewhere the present technology is applied.

FIG. 7 is a diagram for explaining gain calculation according to thepresent technology.

FIG. 8 is a diagram for explaining gain calculation according to thepresent technology.

FIG. 9 is a diagram showing an example structure of a sound processingapparatus.

FIG. 10 is a diagram showing an example structure of a gain calculatingunit.

FIG. 11 is a flowchart for explaining a sound localization controlprocess.

FIG. 12 is a diagram for explaining another method of calculating gainsof speakers.

FIG. 13 is a diagram showing another example structure of a gaincalculating unit.

FIG. 14 is a flowchart for explaining a sound localization controlprocess.

FIG. 15 is a diagram for explaining a method of calculating gains ofspeakers.

FIG. 16 is a diagram showing an example configuration of a computer.

MODES FOR CARRYING OUT THE INVENTION

The following is a description of embodiments to which the presenttechnology is applied, with reference to the drawings.

First Embodiment

<Outline of the Present Technology>

Referring now to FIGS. 1 through 8, an outline of the present technologyis described. In FIGS. 1 through 8, like components are denoted by likereference numerals, and explanation of them will not be unnecessarilyrepeated.

As shown in FIG. 1, a user U11 who is to view or listen to content suchas a moving image accompanied by sound or a music piece is listening tothe sound of two channels being output from two speakers SP1 and SP2 asthe sound of content, for example.

In this case, it is possible to fix a sound image at the position of avirtual sound source VSP1, using position information about the twospeakers SP1 and SP2 outputting the sound of the respective channels.

For example, with the position of the head of the user U11 being theorigin O, the position of the virtual sound source VSP1 in atwo-dimensional coordinate system having the vertical direction and thehorizontal direction as the x-axis direction and the y-axis direction isrepresented by a vector P having the origin O as its starting point inthe drawing.

Since the vector P is a two-dimensional vector, the vector P can berepresented by a linear sum of a vector L₁ and a vector L₂ extending inthe directions of the speaker SP1 and the speaker SP2, respectively,with the origin O being the starting point. That is, the vector P can beexpressed by the following equation (1) using the vector L₁ and thevector L₂:

[Mathematical Formula 1]P=g ₁ L ₁ +g ₂ L ₂  (1)

The coefficient g₁ and the coefficient g₂ by which the vector L₁ and thevector L₂ are multiplied in the equation (1) are calculated. With thesecoefficients g₁ and g₂ being the gains of sounds that are output fromthe speakers SP1 and SP2, respectively, a sound image can be fixed atthe position of the virtual sound source VSP1. That is, a sound imagecan be fixed at the position indicated by the vector P.

The method of controlling the localization position of a sound image bycalculating the coefficients g₁ and g₂ from the position informationabout the two speakers SP1 and SP2 is called two-dimensional VBAP.

In the example shown in FIG. 1, a sound image can be fixed at anyposition on an arc AR11 connecting the speaker SP1 and the speaker SP2.Here, the arc AR11 is part of a circle that passes through therespective positions of the speaker SP1 and the speaker SP2, with theorigin O being the center.

Since the vector P is a two-dimensional vector, if the angle between thevector L₁ and the vector L₂ is greater than 0 degrees but smaller than180 degrees, the coefficients g₁ and g₂ to be the gains are uniquelydetermined. The method of calculating these coefficients g₁ and g₂ isdescribed in detail in the above described Non-Patent Document 1.

In a case where sounds of three channels are to be reproduced, however,the number of speakers that output sounds is three as shown in FIG. 2,for example.

In the example shown in FIG. 2, sounds of the respective channels areoutput from three speakers SP1, SP2, and SP3.

In such a case, the concept is the same as the above describedtwo-dimensional VBAP, except that the number of gains of sounds of therespective channels to be output from the speakers SP1 through SP3, orthe number of the coefficients to be calculated as the gains, is three.

Specifically, in a case where a sound image is to be fixed at theposition of a virtual sound source VSP2, the position of the virtualsound source VSP2 is represented by a three-dimensional vector P havingan origin O as its starting point in a three-dimensional coordinatesystem in which the position of the head of the user U11 is the originO.

Where the three-dimensional vectors extending in the directions from theorigin O as the starting point toward the respective positions of thespeakers SP1 through SP3 are vectors L₁ through L₃, the vector P can beexpressed by a linear sum of the vectors L₁ through L₃ as shown in thefollowing equation (2):[Mathematical Formula 2]P=g ₁ L ₁ +g ₂ L ₂ +g ₃ L ₃  (2)

The coefficient g₁ through g₃ by which the vector L₁ through L₃ aremultiplied in the equation (2) are calculated. With these coefficientsg₁ through g₃ being the gains of sounds that are output from thespeakers SP1 through SP3, respectively, a sound image can be fixed atthe position of the virtual sound source VSP2.

The method of controlling the localization position of a sound image bycalculating the coefficients g₁ through g₃ from the position informationabout the three speakers SP1 through SP3 is called three-dimensionalVBAP.

In the example shown in FIG. 2, a sound image can be fixed at anyposition within a triangular area TR11 on a spherical plane includingthe positions of the speaker SP1, the speaker SP2, and the speaker SP3.Here, the area TR11 is an area that is on the plane of a sphere, has theorigin O as its center, and includes the respective positions of thespeakers SP1 through SP3, and is a triangular area on a spherical planesurrounded by the speakers SP1 through SP3.

With such three-dimensional VBAP, a sound image can be fixed at anyposition in a space.

As shown in FIG. 3, the number of speakers that output sounds isincreased, and areas each equivalent to the triangular area TR11 shownin FIG. 2 are provided in a space so that a sound image can be fixed atany position in those areas, for example.

In the example shown in FIG. 3, five speakers SP1 through SP5 areprovided, and the speakers SP1 through SP5 output sounds of respectivechannels. Here, the speakers SP1 through SP5 are provided on a sphericalplane that has its center at an origin O located at the head of the userU11.

In this case, three-dimensional vectors extending in the directions fromthe origin O toward the respective positions of the speakers SP1 throughSP5 are vectors L₁ through L₅, and the same calculation as the abovedescribed calculation according to equation (2) is performed, todetermine the gains of sounds that are output from the respectivespeakers.

Here, of the areas on the spherical plane having its center at theorigin O, the triangular area surrounded by the speakers SP1, SP4, andSP5 is an area TR21. Likewise, of the areas on the spherical planehaving its center at the origin O, the triangular area surrounded by thespeakers SP3, SP4, and SP5 is an area TR22, and the triangular areasurrounded by the speakers SP2, SP3, and SP5 is an area TR23.

Each of those areas TR21 through TR23 is an area equivalent to the areaTR11 shown in FIG. 2. Where the three-dimensional vector indicating theposition at which a sound image is to be fixed is a vector P, the vectorP indicates a position in the area TR21 in the example shown in FIG. 3.

Therefore, in this example, the same calculation as the calculationaccording to the equation (2) is performed with the vectors L₁, L₄, andL₅ indicating the positions of the speakers SP1, SP4, and SP5, and thegains of sounds that are output from the respective speakers SP1, SP4,and SP5 are calculated. In this case, the gains of sounds that areoutput from the other speakers SP2 and SP3 are 0. That is, no sound isoutput from these speakers SP2 and SP3.

With the five speakers SP1 through SP5 being provided in a space in thismanner, a sound image can be fixed at any position in an area formedwith the areas TR21 through TR23.

As shown in FIG. 4, four speakers SP1 through SP4 might be provided in aspace, and a sound image is to be fixed at the position of a virtualsound source VSP3 located at the center position among these speakersSP1 through SP4.

In the example shown in FIG. 4, the speakers SP1 through SP4 areprovided on the plane of a sphere having its center at an origin O (notshown), and the triangular area that is an area on the plane and issurrounded by the speakers SP1 through SP3 is an area TR31. Of the areason the spherical plane having its center at the origin O, the triangulararea surrounded by the speakers SP2 through SP4 is an area TR32.

The virtual sound source VSP3 is located on the lower right side of thearea TR31. The virtual sound source VSP3 is also located on the upperleft side of the area TR32.

Accordingly, in this case, three-dimensional VBAP should be performedwith respect to the speakers SP1 through SP3, or three-dimensional VBAPshould be performed with respect to the speakers SP2 through SP4. Ineither case, the same three-dimensional VBAP calculation result isobtained, and gains are determined so that only the two speakers SP2 andSP3 output sound, and the remaining speakers SP1 and SP4 do not outputany sound.

By three-dimensional VBAP, in a case where the position at which a soundimage is to be fixed is on the boundary line between triangular areas ona spherical plane connecting three speakers or on a side of a triangleon the spherical plane, only the two speakers located at the two ends ofthe side output sound.

In a case where only the two speakers SP2 and SP3 output sound as above,the user U11 situated in a sweet spot that is an optimum listening spotmight move toward the left in the drawing as indicated by an arrow A11,for example, as shown in FIG. 5.

As a result, the head of the user U11 becomes closer to the speaker SP3,and the sound being output from the speaker SP3 becomes louder to theuser U11. Therefore, the user U11 feels like the virtual sound sourceVSP3, or the sound image, has moved to the lower left as indicated by anarrow A12 in the drawing.

By three-dimensional VBAP, in a case where only two speakers outputsound as shown in FIG. 5, if the user U11 moves away from the sweet spotonly a short distance, the sound image moves in a directionperpendicular to the movement of the user U11. In such a case, the userU11 feels like the sound image has moved in a different direction fromthe direction of his/her movement, and therefore, has a feeling ofstrangeness. That is, the user U11 feels that the localization positionof the sound image is unstable, and the sweet spot range becomesnarrower.

In view of this, according to the present technology as opposed to theabove described VBAP, a larger number of speakers than three, or four ormore speakers, are made to output sound so that the localizationposition of a sound image becomes more stable, and the sweet spot rangebecomes wider accordingly.

Although the number of speakers that are made to output sound may be anynumber equal to or larger than four, explanation of an example casewhere four speakers are made to output sound is continued in thedescription below.

A sound image is to be fixed at the position of a virtual sound sourceVSP3 at the center position among four speakers SP1 through SP4, forexample, as in the example shown in FIG. 4.

According to the present technology, two or three speakers are selectedas a combination in such a case, and VBAP is performed with respect todifferent combinations of speakers, to calculate the gains of soundsthat are output from the four speakers SP1 through SP4.

Accordingly, according to the present technology, all the four speakersSP1 through SP4 output sound as shown in FIG. 6, for example.

In such a case, if a user U11 moves to the left from the sweet spot asshown in an arrow A21 in FIG. 6, the position of the virtual soundsource VSP3 or the localization position of the sound image only movesto the left as shown in an arrow A22 in the drawing. That is, the soundimage does not move downward or in a direction perpendicular to thedirection of movement of the user U11 as in the example shown in FIG. 5,but moves only in the same direction as the direction of movement of theuser U11.

This is because, when the user U11 moves to the left, the user U11becomes closer to the speaker SP3, but the speaker SP1 also exists abovethe speaker SP3. In this case, sound reaches the ears of the user U11from the upper left and the lower left relative to the user U11, andtherefore, the user U11 hardly feels that the sound image has moveddownward.

Accordingly, the localization position of the sound image can be mademore stable than by a conventional VBAP technique, and as a result, thesweet spot range can be made wider.

Next, the control on localization of sound according to the presenttechnology is described in greater detail.

In the present technique, a vector that indicates a position at which asound image is to be fixed is a vector P that has its starting point atthe origin O (not shown) of a three-dimensional coordinate system, andthe vector P is expressed by the following equation (3):[Mathematical Formula 3]P=g ₁ L ₁ +g ₂ L ₂ +g ₃ L ₃ +g ₄ L ₄  (3)

In the equation (3), the vectors L₁ through L₄ indicatethree-dimensional vectors extending toward the positions of speakers SP1through SP4 that are located in the vicinity of the localizationposition of the sound image and are arranged to surround thelocalization position of the sound image. Also, in the equation (3), g₁through g₄ represent the coefficients that are to be calculated as thegains of sounds of the respective channels to be output from thespeakers SP1 through SP4.

In the equation (3), the vector P is expressed as a linear sum of thefour vectors L₁ through L₄. Since the vector P is a three-dimensionalvector, the four coefficients g₁ through g₄ are not uniquely determined.

Therefore, according to the present technology, the respectivecoefficients g₁ through g₄ to be gains are calculated by the methoddescribed below.

A sound image is to be fixed at the center position of a tetragon on aspherical plane surrounded by the four speakers SP1 through SP4 shown inFIG. 4, or at the position of the virtual sound source VSP3.

First, one of the tetragon on the spherical plane having the speakersSP1 through SP4 as the corners is selected, and a virtual speaker(hereinafter referred to as “the virtual speaker”) is assumed to existon the side.

As shown in FIG. 7, in the tetragon on the spherical plane having thespeakers SP1 through SP4 at the corners, the side connecting thespeakers SP3 and SP4 located at the lower left corner and the lowerright corner in the drawing is selected, for example. The virtualspeaker VSP′ is assumed to exist at the position of the intersectionpoint between the side connecting the speakers SP3 and SP4 and aperpendicular extending from the position of the virtual sound sourceVSP3, for example.

Three-dimensional VBAP is then performed with respect to the threespeakers: the virtual speaker VSP′, and the speakers SP1 and SP2 at theupper left corner and the upper right corner in the drawing.Specifically, a calculation is performed according to the same equationas the above described equation (2), to determine a coefficient g₁, acoefficient g₂, and a coefficient g′, which are to be the gains ofsounds to be output from the speaker SP1, the speaker SP2, and thevirtual speaker VSP′.

In FIG. 7, a vector P is expressed as a linear sum of three vectorsextending from the origin O, or of the vector L₁ extending toward thespeaker SP1, the vector L₂ extending toward the speaker SP2, and thevector L′ extending toward the virtual speaker VSP′. That is, the vectorP is expressed as P=g₁L₁+g₂L₂+g′L′.

Here, to fix a sound image at the position of the virtual sound sourceVSP3, the virtual speaker VSP′ needs to output sound with the gain g′,but the virtual speaker VSP′ does not actually exist. Therefore, asshown in FIG. 8, the two speakers SP3 and SP4 at the two ends of theside on which the virtual speaker VSP′ is located in the tetragon areused to fix the sound image at the position of the virtual speaker VSP′according to the present technology. In this manner, the virtual speakerVSP′ is realized.

Specifically, two-dimensional VBAP is performed with respect to the twospeakers SP3 and SP4, which are located at the two ends of the side onwhich the virtual speaker VSP′ is located on the spherical plane. Thatis, a calculation is performed according to the same equation as theabove described equation (1), to calculate a coefficient g₃′ and acoefficient g₄′, which are to be the gains of sounds to be output fromthe speaker SP3 and the speaker SP4, respectively.

In the example shown in FIG. 8, the vector L′ extending toward thevirtual speaker VSP′ is expressed as a linear sum of the vector L₃extending toward the speaker SP3 and the vector L₄ extending toward thespeaker SP4. That is, the vector L′ is expressed as L′=g₃′L₃+g₄′L₄.

The value g′g₃′ obtained by multiplying the calculated coefficient g₃′by the coefficient g′ is set as the gain of the sound to be output fromthe speaker SP3, and the value g′g₄′ obtained by multiplying thecalculated coefficient g₄′ by the coefficient g′ is set as the gain ofthe sound to be output from the speaker SP4. In this manner, the virtualspeaker VSP′ that outputs sound with the gain g′ is realized by thespeakers SP3 and SP4.

Here, the value of g′g₃′ to be a gain value is the value of thecoefficient g₃ in the above described equation (3), and the value ofg′g₄′ to be a gain value is the value of the coefficient g₄ in the abovedescribed equation (3).

The values g₁, g₂, g′g₃′, and g′g₄′, which have been obtained as aboveand are not 0, are set as the gains of sounds of the respective channelsto be output from the speakers SP1 through SP4. In this manner, the fourspeakers are made to output sound, and the sound image can be fixed atthe target position.

As the sound image is fixed at a position by causing the four speakersto output sound as above, the localization position of the sound imagecan be made more stable than a sound image fixed at a position by aconventional VBAP technique. Accordingly, the sweet spot range can bewidened.

<Example Structure of a Sound Processing Apparatus>

Next, a specific example to which the above described present technologyis applied is described. FIG. 9 is a diagram showing an examplestructure of an embodiment of a sound processing apparatus to which thepresent technology is applied.

A sound processing apparatus 11 performs gain adjustment on a monauralsound signal supplied from outside for respective channels. By doing so,the sound processing apparatus 11 generates sound signals of N channels(N≥5), and supplies the sound signals to speakers 12-1 through 12-Ncorresponding to the N channels.

The speakers 12-1 through 12-N output sounds of the respective channelsbased on the sound signals supplied from the sound processing apparatus11. That is, the speakers 12-1 through 12-N are sound outputting unitsserving as sound sources that output sounds of the respective channels.Hereinafter, when there is no particular need to distinguish thespeakers 12-1 through 12-N from one another, the speakers 12-1 through12-N will be also referred to simply as the speakers 12. Although thespeakers 12 are not included in the sound processing apparatus 11 inFIG. 9, the speakers 12 may be included in the sound processingapparatus 11. Also, the respective components constituting the soundprocessing apparatus 11 and the speakers 12 may be provided in severalapparatuses, to form a sound processing system including the respectivecomponents of the sound processing apparatus 11 and the speakers 12.

The speakers 12 are arranged to surround the position at which a user isassumed to exist when viewing and listening to content or the like (theposition will be hereinafter also referred to simply as the userposition). For example, the respective speakers 12 are arranged atpositions on the plane of a sphere having its center at the userposition. In other words, the respective speakers 12 are arranged atpositions at the same distance from the user. Also, sound signals may besupplied from the sound processing apparatus 11 to the speakers 12 in awired or wireless manner.

The sound processing apparatus 11 includes a speaker selecting unit 21,a gain calculating unit 22, a gain determining unit 23, a gainoutputting unit 24, and a gain adjusting unit 25.

Sound signals of sound collected by a microphone attached to an objectsuch as a mobile object, and position information about the object aresupplied to the sound processing apparatus 11.

Based on the object's position information supplied from outside, thespeaker selecting unit 21 identifies the position at which a sound imageof sound to be emitted from the object in the space including thespeakers 12 is to be fixed (the position to be identified will behereinafter also referred to as the target sound image position), andsupplies the result of the identification to the gain calculating unit22.

Based on the target sound image position, the speaker selecting unit 21selects four speakers 12 to output sound as processing-target speakers12 from among the N speakers 12, and supplies select informationindicating the result of the selection to the gain calculating unit 22,the gain determining unit 23, and the gain outputting unit 24.

The gain calculating unit 22 calculates gains of the processing-targetspeakers 12 based on the select information supplied from the speakerselecting unit 21 and the target sound image position, and supplies thegains to the gain outputting unit 24. The gain determining unit 23determines gains of the speakers 12 other than the processing-targetspeakers based on the select information supplied from the speakerselecting unit 21, and supplies the gains to the gain outputting unit24. For example, the gains of the speakers 12 that are not theprocessing targets are set at “0”. That is, the speakers 12 that are notthe processing targets are controlled not to output any sound of theobject.

The gain outputting unit 24 supplies the N gains supplied from the gaincalculating unit 22 and the gain determining unit 23, to the gainadjusting unit 25. At this point, the gain outputting unit 24 determinesthe supply destinations of the N gains supplied from the gaincalculating unit 22 and the gain determining unit 23 in the gainadjusting unit 25 based on the select information supplied from thespeaker selecting unit 21.

Based on the respective gains supplied from the gain outputting unit 24,the gain adjusting unit 25 performs gain adjustment on the object'ssound signal supplied from outside, and supplies the resultant soundsignals of the N channels to the speakers 12, so that the speakers 12output sound.

The gain adjusting unit 25 includes amplifiers 31-1 through 31-N. Basedon the gains supplied from the gain outputting unit 24, the amplifiers31-1 through 31-N perform gain adjustment on the sound signal suppliedfrom outside, and supply the resultant sound signals to the speakers12-1 through 12-N.

Hereinafter, when there is no need to distinguish the amplifiers 31-1through 31-N from one another, the amplifiers 31-1 through 31-N will bealso referred to simply as the amplifiers 31.

<Example Structure of the Gain Calculating Unit>

The gain calculating unit 22 shown in FIG. 9 has the structure shown inFIG. 10, for example.

The gain calculating unit 22 shown in FIG. 10 includes a virtual speakerposition determining unit 61, a three-dimensional gain calculating unit62, a two-dimensional gain calculating unit 63, a multiplier 64, and amultiplier 65.

The virtual speaker position determining unit 61 determines the positionof a virtual speaker based on information indicating the target soundimage position and the select information supplied from the speakerselecting unit 21. The virtual speaker position determining unit 61supplies the information indicating the target sound image position, theselect information, and information indicating the position of thevirtual speaker to the three-dimensional gain calculating unit 62, andsupplies the select information and the information indicating theposition of the virtual speaker to the two-dimensional gain calculatingunit 63.

Based on the respective pieces of the information supplied from thevirtual speaker position determining unit 61, the three-dimensional gaincalculating unit 62 performs three-dimensional VBAP with respect to twospeakers 12 of the processing-target speakers 12, and the virtualspeaker. The three-dimensional gain calculating unit 62 then suppliesthe gains of the two speakers 12 obtained through the three-dimensionalVBAP to the gain outputting unit 24, and supplies the gain of thevirtual speaker to the multiplier 64 and the multiplier 65.

Based on the respective pieces of the information supplied from thevirtual speaker position determining unit 61, the two-dimensional gaincalculating unit 63 performs two-dimensional VBAP with respect to twospeakers 12 of the processing-target speakers 12, and supplies theresultant gains of the speakers 12 to the multiplier 64 and themultiplier 65.

The multiplier 64 obtains ultimate gains of the speakers 12 bymultiplying the gains supplied from the two-dimensional gain calculatingunit 63 by the gains supplied from the three-dimensional gaincalculating unit 62, and supplies the ultimate gains to the gainoutputting unit 24. The multiplier 65 obtains ultimate gains of thespeakers 12 by multiplying the gain supplied from the two-dimensionalgain calculating unit 63 by the gains supplied from thethree-dimensional gain calculating unit 62, and supplies the ultimategains to the gain outputting unit 24.

<Description of a Sound Localization Control Process>

Meanwhile, when object's position information and an object's soundsignal are supplied to the sound processing apparatus 11, and aninstruction to output sound of the object is issued, the soundprocessing apparatus 11 starts a sound localization control process, tocause sound of the object to be output, and fix the sound image at anappropriate position.

Referring now to the flowchart in FIG. 11, the sound localizationcontrol process to be performed by the sound processing apparatus 11 isdescribed.

In step S11, the speaker selecting unit 21 selects processing-targetspeakers 12 based on the object's position information supplied fromoutside.

Specifically, the speaker selecting unit 21 identifies a target soundimage position based on the object's position information, for example,and selects, from among the N speakers 12, the processing-targetspeakers 12 that are four speakers 12 that are located in the vicinityof the target sound image position and are arranged to surround thetarget sound image position.

In a case where the position of the virtual sound source VSP3 shown inFIG. 7 is set as the target sound image position, for example, thespeakers 12 equivalent to the four speakers SP1 through SP4 surroundingthe virtual sound source VSP3 are selected as the processing-targetspeakers 12.

The speaker selecting unit 21 supplies the information indicating thetarget sound image position to the virtual speaker position determiningunit 61, and supplies the select information indicating the fourprocessing-target speakers 12 to the virtual speaker positiondetermining unit 61, the gain determining unit 23, and the gainoutputting unit 24.

In step S12, the virtual speaker position determining unit 61 determinesthe position of the virtual speaker based on the information indicatingthe target sound image position and the select information supplied fromthe speaker selecting unit 21. As in the example shown in FIG. 7, forexample, the position of the intersection point between the sideconnecting the speakers 12 located at the lower left corner and thelower right corner relative to the user among the processing-targetspeakers 12 on a spherical plane, and the perpendicular extending fromthe target sound image position toward the side is set as the positionof the virtual speaker.

After the position of the virtual speaker is determined, the virtualspeaker position determining unit 61 supplies the information indicatingthe target sound image position, the select information, and theinformation indicating the position of the virtual speaker to thethree-dimensional gain calculating unit 62, and supplies the selectinformation and the information indicating the position of the virtualspeaker to the two-dimensional gain calculating unit 63.

The position of the virtual speaker may be any position, as long as itis located on a side of a tetragon on the spherical plane, the tetragonhaving the four processing-target speakers 12 at the respective corners.Even in a case where the number of processing-target speakers 12 is fiveor larger, the position of the virtual speaker may be any position on aside of a polygon on the spherical plane, the polygon having thoseprocessing-target speakers 12 at the corners.

In step S13, the three-dimensional gain calculating unit 62 calculatesgains for the virtual speaker and two processing-target speakers 12based on the information indicating the target sound image position, theselect information, and the information about the position of thevirtual speaker, the information having been supplied from the virtualspeaker position determining unit 61.

Specifically, the three-dimensional gain calculating unit 62 determinesthe vector P to be the three-dimensional vector indicating the targetsound image position, and determines the vector L′ to be thethree-dimensional vector extending toward the virtual speaker. Thethree-dimensional gain calculating unit 62 also determines the vector L₁to be the vector extending toward the speaker 12 in the same positionalrelationship as the speaker SP1 shown in FIG. 7 among theprocessing-target speakers 12, and the vector L₂ to be the vectorextending toward the speaker 12 in the same positional relationship asthe speaker SP2.

The three-dimensional gain calculating unit 62 then determines theequation expressing the vector P as the linear sum of the vector L′, thevector L₁, and the vector L₂, and performs calculation according to theequation, to calculate gains that are the coefficient g′, thecoefficient g₁, and the coefficient g₂ of the vector L′, the vector L₁,and the vector L₂, respectively. That is, the same calculation as theabove described calculation according to the equation (2) is performed.

The three-dimensional gain calculating unit 62 supplies the calculatedcoefficients g₁ and g₂ of the speakers 12 in the same positionalrelationships as the speakers SP1 and SP2, as the gains of the sounds tobe output from these speakers 12, to the gain outputting unit 24.

The three-dimensional gain calculating unit 62 also supplies thecalculated coefficient g′ of the virtual speaker, as the gain of thesound to be output from the virtual speaker, to the multiplier 64 andthe multiplier 65.

In step S14, the two-dimensional gain calculating unit 63 calculatesgains for two processing-target speakers 12 based on the selectinformation and the information about the position of the virtualspeaker, the information having been supplied from the virtual speakerposition determining unit 61.

Specifically, the two-dimensional gain calculating unit 63 determinesthe vector L′ to be the three-dimensional vector indicating the positionof the virtual speaker. The two-dimensional gain calculating unit 63also determines the vector L₃ to be the vector extending toward thespeaker 12 in the same positional relationship as the speaker SP3 shownin FIG. 8 among the processing-target speakers 12, and the vector L₄ tobe the vector extending toward the speaker 12 in the same positionalrelationship as the speaker SP4.

The two-dimensional gain calculating unit 63 then determines theequation expressing the vector L′ as the linear sum of the vector L₃ andthe vector L₄, and performs calculation according to the equation, tocalculate gains that are the coefficient g₃′ and the coefficient g₄′ ofthe vector L₃ and the vector L₄, respectively. That is, the samecalculation as the above described calculation according to the equation(1) is performed.

The two-dimensional gain calculating unit 63 supplies the calculatedcoefficients g₃′ and g₄′ of the speakers 12 in the same positionalrelationships as the speakers SP3 and SP4, as the gains of the sounds tobe output from these speakers 12, to the multiplier 64 and themultiplier 65.

In step S15, the multiplier 64 and the multiplier 65 multiply the gainsg₃′ and g₄′ supplied from the two-dimensional gain calculating unit 63by the gain g′ of the virtual speaker supplied from thethree-dimensional gain calculating unit 62, and supplies the resultantgains to the gain outputting unit 24.

Accordingly, as the ultimate gain of the speaker 12 in the samepositional relationship as the speaker SP3 shown in FIG. 8 among thefour processing-target speakers 12, the gain g₃=g′g₃′ is supplied to thegain outputting unit 24. Likewise, as the ultimate gain of the speaker12 in the same positional relationship as the speaker SP4 shown in FIG.8 among the four processing-target speakers 12, the gain g₄=g′g₄′ issupplied to the gain outputting unit 24.

In step S16, the gain determining unit 23 determines gains of thespeakers 12 other than the processing-target speakers based on theselect information supplied from the speaker selecting unit 21, andsupplies the gains to the gain outputting unit 24. For example, thegains of all the speakers 12 that are not the processing targets are setat “0”.

After the gains g₁, g₂, g′g₃′, and g′g₄′ from the gain calculating unit22, and the gain “0” from the gain determining unit 23 are supplied tothe gain outputting unit 24, the gain outputting unit 24 supplies thesegains to the amplifiers 31 of the gain adjusting unit 25 based on theselect information supplied from the speaker selecting unit 21.

Specifically, the gain outputting unit 24 supplies the gains g₁, g₂,g′g₃′, and g′g₄′ to the amplifiers 31 that supply sound signals to therespective processing-target speakers 12, or the respective speakers 12equivalent to the speakers SP1 through SP4 shown in FIG. 7. For example,in a case where the speaker 12 equivalent to the speaker SP1 is thespeaker 12-1, the gain outputting unit 24 supplies the gain g₁ to theamplifier 31-1.

The gain outputting unit 24 also supplies the gain “0” supplied from thegain determining unit 23, to the amplifiers 31 that supply sound signalsto the speakers 12 that are not processing targets.

In step S17, the amplifiers 31 of the gain adjusting unit 25 performgain adjustment on an object's sound signal supplied from outside basedon the gains supplied from the gain outputting unit 24, and supply theresultant sound signals to the speakers 12, so that the speakers 12output sound.

The respective speakers 12 output sound based on the sound signalssupplied from the amplifiers 31. More specifically, only the fourprocessing-target speakers 12 output sound. Accordingly, a sound imagecan be fixed at a target position. As sound is output from the speakers12, the sound localization control process comes to an end.

As described above, the sound processing apparatus 11 selects fourspeakers 12 as processing targets based on position information about anobject, and performs VBAP with respect to a combination of two or threespeakers among those speakers 12 and a virtual speaker. The soundprocessing apparatus 11 then performs gain adjustment on a sound signalbased on the gains of the respective processing-target speakers 12, thegains having been obtained through VBAP performed on differentcombinations.

Accordingly, sound can be output from the four speakers 12 surroundingthe target sound image position, and localization of sound can be mademore stable. As a result, the sweet spot range can be made wider.

Second Embodiment

<Gain Calculation>

In the above described example, two or three speakers to form onecombination of speakers are selected from among five speakers includinga virtual speaker, and VBAP is performed on more than one combination,to calculate gains of the processing-target speakers 12. According tothe present technology, however, it is also possible to calculate gainsby selecting combinations from among four processing-target speakers 12,and performing VBAP on each of the combinations, without any virtualspeaker determined.

In such a case, the number of times VBAP should be performed variesdepending on the target sound image position, as shown in FIG. 12, forexample. In FIG. 12, the components equivalent to those shown in FIG. 7are denoted by the same reference numerals as those used in FIG. 7, andexplanation of them will not be unnecessarily repeated.

In a case where the position of a virtual sound source, or a targetsound image position, is the position indicated by an arrow Q11, forexample, the position indicated by the arrow Q11 is in a triangular areasurrounded by a speaker SP1, a speaker SP2, and a speaker SP4 on aspherical plane. Accordingly, if three-dimensional VBAP is performedwith respect to the combination of speakers consisting of the speakersSP1, SP2, and SP4 (this combination will be hereinafter also referred toas the first combination), the gains of sounds to be output from thethree speakers SP1, SP2, and SP4 are determined.

Meanwhile, the position indicated by the arrow Q11 is also a position ina triangular area surrounded by the speaker SP2, a speaker SP3, and thespeaker SP4 on the spherical plane. Accordingly, if three-dimensionalVBAP is performed with respect to the combination of speakers consistingof the speakers SP2, SP3, and SP4 (this combination will be hereinafteralso referred to as the second combination), the gains of sounds to beoutput from the three speakers SP2, SP3, and SP4 are determined.

Here, each of the gains of the speakers not used in the firstcombinations and the second combinations is set at “0”, so that two setsof gains can be obtained as the respective gains of the four speakersSP1 through SP4 in the first and second combinations.

For each of the speakers, the sum of the gains of the speaker obtainedin the first and the second combinations is obtained as a gain sum. Forexample, where the gain of the speaker SP1 in the first combination isg₁(1), and the gain of the speaker SP1 in the second combination isg₁(2), the gain sum g_(s1) of the speaker SP1 is expressed as the gainsum g_(s1)=g₁(1)+g₁(2).

Since the speaker SP1 is not included in the second combination, g₁(2)is 0. Since the speaker SP1 is included in the first combination ofspeakers, g₁(1) is not 0. as a result, the gain sum g_(s1) of thespeaker SP1 is not 0. The same applies to the gain sums of the otherspeakers SP2 through SP4.

As the gain sums of the respective speakers are determined in the abovemanner, the values obtained by normalizing the gain sums of therespective speakers with the sum of squares of these gain sums are setas the ultimate gains of these speakers, or more specifically, as thegains of the sounds to be output form the speakers.

As the gains of the respective speakers SP1 through SP4 are determinedin the above manner, gains that are not 0 are invariably obtained.Accordingly, each of the four speakers SP1 through SP4 can be made tooutput sound, and a sound image can be fixed at a desired position.

In the description below, the gain of a speaker SPk (1≤k≤4) in a mthcombination (1≤m≤4) is represented by g_(k)(m). The gain sum of aspeaker SPk (1≤k≤4) is represented by g_(sk).

Further, in a case where the target sound image position is the positionindicated by an arrow Q12, or the position of the intersection pointbetween the line connecting the speaker SP2 and the speaker SP3 and theline connecting the speaker SP1 and the speaker SP4 on the sphericalplane, there are four combinations of three speakers.

Specifically, where there is the combination of the speaker SP1, thespeaker SP2, and the speaker SP3 (this combination will be hereinafterreferred to as the first combination), the combination of the speakerSP1, the speaker SP2, and the speaker SP4 (this combination will behereinafter referred to as the second combination) is possible. Also,where there is the combination of the speaker SP1, the speaker SP3, andthe speaker SP4 (this combination will be hereinafter referred to as thethird combination), the combination of the speaker SP2, the speaker SP3,and the speaker SP4 (this combination will be hereinafter referred to asthe fourth combination) is possible.

In this case, three-dimensional VBAP is performed on each of the firstthrough fourth combinations, to determine the gains of the respectivespeakers. The sum of the four gains obtained with respect to the samespeaker is set as a gain sum, and the values obtained by normalizing thegain sums of the respective speakers with the sum of squares of the fourgain sums determined with respect to the respective speakers are set asthe ultimate gains of these speakers.

In a case where the target sound image position is the positionindicated by the arrow Q12, if the tetragon formed with the speakers SP1through SP4 on the spherical plane is a rectangle or the like, the sameresult of calculation as three-dimensional VBAP is obtained from thefirst combination and the fourth combination, for example. Accordingly,if three-dimensional VBAP is performed with respect to appropriate twocombinations such as the first and second combinations in this case, thegains of the respective speakers can be obtained. However, in a casewhere the tetragon formed with the speakers SP1 through SP4 on thespherical plane is not a rectangle or the like but an asymmetrictetragon, it is necessary to perform three-dimensional VBAP with respectto each of the four combinations.

<Example Structure of the Gain Calculating Unit>

In a case where gains are calculated by selecting combinations ofspeakers from among four processing-target speakers 12 and performingVBAP with respect to each of those combinations, without any virtualspeaker determined, the gain calculating unit 22 shown in FIG. 9 has thestructure shown in FIG. 13, for example.

The gain calculating unit 22 shown in FIG. 13 includes a selecting unit91, a three-dimensional gain calculating unit 92-1, a three-dimensionalgain calculating unit 92-2, a three-dimensional gain calculating unit92-3, a three-dimensional gain calculating unit 92-4, and an adder 93.

Based on information indicating the target sound image position andselect information supplied from a speaker selecting unit 21, theselecting unit 91 determines combinations of three speakers 12surrounding the target sound image position from among four speakers 12selected as the processing targets. The selecting unit 91 suppliesinformation indicating the combinations of speakers 12 and theinformation indicating the target sound image position to thethree-dimensional gain calculating units 92-1 through 92-4.

The three-dimensional gain calculating units 92-1 through 92-4 performthree-dimensional VBAP based on the information indicating thecombinations of speakers 12 and the information indicating the targetsound image position supplied from the selecting unit 91, and supply theresultant gains of the respective speakers 12 to the adder 93. In thedescription below, where there is no particular need to distinguish thethree-dimensional gain calculating units 92-1 through 92-4 from oneanother, the three-dimensional gain calculating units 92-1 through 92-4will be also referred to simply as the three-dimensional gaincalculating units 92.

The adder 93 determines gain sums based on the gains of the respectiveprocessing-target speakers 12 supplied from the three-dimensional gaincalculating units 92-1 through 92-4, and calculates the ultimate gainsof the respective processing-target speakers 12 by normalizing thesegain sums. The adder 93 then supplies the ultimate gains to the gainoutputting unit 24.

<Description of a Sound Localization Control Process>

Referring now to the flowchart in FIG. 14, a sound localization controlprocess to be performed in a case where the gain calculating unit 22 hasthe structure shown in FIG. 13 is described.

The procedure in step S41 is the same as the procedure in step S11 inFIG. 11, and therefore, explanation thereof is not repeated herein.

In step S42, the selecting unit 91 determines combinations of speakers12 based on the information indicating the target sound image positionand the select information supplied from the speaker selecting unit 21,and supplies information indicating the combinations of speakers 12 andthe information indicating the target sound image position to thethree-dimensional gain calculating units 92.

In a case where the target sound image position is the positionindicated by the arrow Q11 in FIG. 12, for example, the combination (thefirst combination) of the speakers 12 including the three speakers 12equivalent to the speakers SP1, SP2, and SP4 is determined. Also, thecombination (the second combination) of the speakers 12 including thethree speakers 12 equivalent to the speakers SP2, SP3, and SP4 isdetermined.

In this case, the selecting unit 91 supplies information indicating thefirst combination of speakers 12 and the information indicating thetarget sound image position to the three-dimensional gain calculatingunit 92-1, and supplies information indicating the second combination ofspeakers 12 and the information indicating the target sound imageposition to the three-dimensional gain calculating unit 92-2. In thiscase, the information indicating the combinations of speakers 12 and thelike are not supplied to the three-dimensional gain calculating units92-3 and 92-4, and the three-dimensional gain calculating units 92-3 and92-4 do not perform any calculation of three-dimensional VBAP, either.

In step S43, based on the information indicating the combinations ofspeakers 12 and the information indicating the target sound imageposition supplied from the selecting unit 91, the three-dimensional gaincalculating units 92 calculate gains of the respective speakers 12 inthe combinations of speakers 12, and supply the gains to the adder 93.

Specifically, the three-dimensional gain calculating units 92 determinegains of the respective speakers 12 by carrying out the same procedureas that in step S13 in FIG. 11 with respect to the three speakers 12indicated by the information indicating the combination of speakers 12.That is, the same calculation as the above described calculationaccording to the equation (2) is performed. The gain of the remainingone speaker 12 other than the three speakers 12 indicated by theinformation indicating the combination of speakers 12 among the fourprocessing-target speakers 12 is set at “0”.

In a case where the two combinations, the first and second combinations,are determined in step S42, for example, the three-dimensional gaincalculating unit 92-1 calculates gains of the respective speakers 12through three-dimensional VBAP with respect to the first combination.The three-dimensional gain calculating unit 92-2 calculates gains of therespective speakers 12 through three-dimensional VBAP with respect tothe second combination.

Specifically, the combination of the speakers including the threespeakers 12 equivalent to the speakers SP1, SP2, and SP4 shown in FIG.12 is determined as the first combination. In this case, thethree-dimensional gain calculating unit 92-1 calculates the gain g₁(1)of the speaker 12 equivalent to the speaker SP1, the gain g₂(1) of thespeaker 12 equivalent to the speaker SP2, and the gain g₄(1) of thespeaker 12 equivalent to the speaker SP4. Meanwhile, the gain g₃(1) ofthe speaker 12 equivalent to the speaker SP3 is set at “0”.

In step S44, the adder 93 calculates ultimate gains of theprocessing-target speakers 12 based on the gains of the respectivespeakers 12 supplied from the three-dimensional gain calculating units92, and supplies the ultimate gains to the gain outputting unit 24.

For example, the adder 93 calculates the gain sum g_(s1) of the speaker12 equivalent to the speaker SP1 by determining the sum of the gainsg₁(1), g₁(2), g₁(3), and g₁(4) of the speaker 12, the gains having beensupplied from the three-dimensional gain calculating units 92. Likewise,the adder 93 also calculates the gain sum g_(s2) of the speaker 12equivalent to the speaker SP2, the gain sum g_(s3) of the speaker 12equivalent to the speaker SP3, and the gain sum g_(s4) of the speaker 12equivalent to the speaker SP4.

The adder 93 then determines the ultimate gain g₁ (coefficient g₁) ofthe speaker 12 equivalent to the speaker SP1 by normalizing the gain sumg_(s1) of the speaker 12 equivalent to the speaker SP1 with the sum ofsquares of the gain sums g_(s1) through g_(s4). The adder 93 alsodetermines the ultimate gains g₂ through g₄ of the speakers 12equivalent to the speakers SP2 through SP4 through the same calculationas above.

After the gains of the processing-target speakers 12 are determined inthe above manner, the procedures in steps S45 and S46 are carried out,and the sound localization control process then comes to an end.However, these procedures are the same as the procedures in steps S16and S17 in FIG. 11, and therefore, explanation of them is not repeatedherein.

As described above, the sound processing apparatus 11 selects fourspeakers 12 as processing targets based on position information about anobject, and performs VBAP with respect to combinations of speakers 12that are three speakers 12 among the four processing-target speakers 12.The sound processing apparatus 11 then determines the ultimate gains ofthe respective processing-target speakers 12 by determining the sum ofgains of each speaker 12 obtained through VBAP performed with respect todifferent combinations, and performs gain adjustment on sound signals.

Accordingly, sound can be output from the four speakers 12 surroundingthe target sound image position, and localization of sound can be mademore stable. As a result, the sweet spot range can be made wider.

Although an example case where four speakers 12 surrounding the targetsound image position are the processing-target speakers 12 has beendescribed in this embodiment, the number of speakers 12 to be selectedas the processing targets may be four or larger.

For example, in a case where five speakers 12 are selected as theprocessing-target speakers 12, the combination of speakers 12 consistingof three speakers 12 surrounding the target sound image position isselected as one combination among the five speakers 12.

Specifically, as shown in FIG. 15, speakers 12 equivalent to fivespeakers SP1 through SP5 are selected as processing-target speakers 12,and the target sound image position is the position indicated by anarrow Q21.

In this case, the combination of the speakers SP1, SP2, and SP3 isselected as a first combination, and the combination of the speakersSP1, SP2, and SP4 is selected as a second combination. Also, thecombination of the speakers SP1, SP2, and SP5 is selected as a thirdcombination.

Gains of the respective speakers are determined with respect to thefirst through third combinations, and ultimate gains are calculated fromthe gain sums of the respective speakers. That is, with respect to thefirst through third combinations, the procedure in step S43 in FIG. 14is carried out, and the procedures in steps S44 through S46 are thencarried out.

As described above, in a case where five or more speakers 12 areselected as the processing-target speakers 12, sound is also output fromall the processing-target speakers 12, so that a sound image can befixed.

The above described series of processes may be performed by hardware ormay be performed by software. Where the series of processes are to beperformed by software, the program that forms the software is installedinto a computer. Here, the computer may be a computer incorporated intospecial-purpose hardware, or may be a general-purpose computer that canexecute various kinds of functions as various kinds of programs areinstalled thereinto.

FIG. 16 is a block diagram showing an example structure of the hardwareof a computer that performs the above described series of processes inaccordance with programs.

In the computer, a CPU 801, a ROM 802, and a RAM 803 are connected toone another by a bus 804.

An input/output interface 805 is further connected to the bus 804. Aninput unit 806, an output unit 807, a recording unit 808, acommunication unit 809, and a drive 810 are connected to theinput/output interface 805.

The input unit 806 is formed with a keyboard, a mouse, a microphone, animaging device, and the like. The output unit 807 is formed with adisplay, a speaker, and the like. The recording unit 808 is formed witha hard disk, a nonvolatile memory, or the like. The communication unit809 is formed with a network interface or the like. The drive 810 drivesa removable medium 811 such as a magnetic disk, an optical disk, amagnetooptical disk, or a semiconductor memory.

In the computer having the above described structure, the CPU 801 loadsa program recorded in the recording unit 808 into the RAM 803 via theinput/output interface 805 and the bus 804, for example, and executesthe program, so that the above described series of processes areperformed.

The program to be executed by the computer (the CPU 801) may be recordedon the removable medium 811 as a package medium to be provided, forexample. Alternatively, the program can be provided via a wired orwireless transmission medium such as a local area network, the Internet,or digital satellite broadcasting.

In the computer, the program can be installed into the recording unit808 via the input/output interface 805 when the removable medium 811 ismounted on the drive 810. The program can also be received by thecommunication unit 809 via a wired or wireless transmission medium, andbe installed into the recording unit 808. Alternatively, the program maybe installed beforehand into the ROM 802 or the recording unit 808.

The program to be executed by the computer may be a program forperforming processes in chronological order in accordance with thesequence described in this specification, or may be a program forperforming processes in parallel or performing a process when necessary,such as when there is a call.

It should be noted that embodiments of the present technology are notlimited to the above described embodiments, and various modificationsmay be made to them without departing from the scope of the presenttechnology.

For example, the present technology can be embodied in a cloud computingstructure in which one function is shared among apparatuses via anetwork, and processing is performed by the apparatuses cooperating withone another.

The respective steps described with reference to the above describedflowcharts can be carried out by one device or can be shared amongdevices.

In a case where more than one process is included in one step, theprocesses included in the step can be performed by one device or can beshared among devices.

Further, the present technology may take the following forms.

[1]

A sound processing apparatus including:

a gain calculating unit that determines output gains of sounds to beoutput from four or more sound outputting units located close to a soundimage localization position as a target position by calculating gains ofsounds to be output from the sound outputting units based on apositional relationship between the sound outputting units with respectto each of different combinations among combinations of two or three ofthe four or more sound outputting units, the output gains being to beused for fixing a sound image at the sound image localization position;and

a gain adjusting unit that performs gain adjustment on sounds to beoutput from the sound outputting units based on the output gains.

[2]

The sound processing apparatus of [1], wherein at least four of theoutput gains each have a value other than 0.

[3]

The sound processing apparatus of [1] or [2], wherein the gaincalculating unit includes:

a first gain calculating unit that calculates output gains of a virtualsound outputting unit and two of the sound outputting units based on apositional relationship among the virtual sound outputting unit, the twoof the sound outputting units, and the sound image localizationposition;

a second gain calculating unit that calculates gains of other two of thesound outputting units than the two of the sound outputting units basedon a positional relationship among the other two of the sound outputtingunits and the virtual sound outputting unit, the gains of the other twoof the sound outputting units being to be used for fixing a sound imageat a position of the virtual sound outputting unit; and

a calculating unit that calculates output gains of the other two of thesound outputting units based on the gains of the other two of the soundoutputting units and the output gain of the virtual sound outputtingunit.

[4]

The sound processing apparatus of [3], wherein the calculating unitcalculates the output gains of the other two of the sound outputtingunits by multiplying the gains of the other two of the sound outputtingunits by the output gain of the virtual sound outputting unit.

[5]

The sound processing apparatus of [3] or [4], wherein the position ofthe virtual sound outputting unit is on a side of a polygon having thefour or more sound outputting units at the corners thereof.

[6]

The sound processing apparatus of [1] or [2], wherein the gaincalculating unit includes:

a virtual gain calculating unit that calculates output gains of three ofthe sound outputting units based on a positional relationship among thethree of the sound outputting units and the sound image localizationposition; and

a calculating unit that calculates ultimate output gains of the soundoutputting units based on the output gains calculated by the virtualgain calculating unit calculating the output gains with respect todifferent combinations among the combinations.

[7]

The sound processing apparatus of [6], wherein the calculating unitcalculates an ultimate output gain of the same one of the soundoutputting units by determining a sum of the output gains determinedwith respect to the same one of the sound outputting units.

REFERENCE SIGNS LIST

-   11 Sound processing apparatus-   12-1 through 12-N, and 12 Speaker-   21 Speaker selecting unit-   22 Gain calculating unit-   25 Gain adjusting unit-   61 Virtual speaker position determining unit-   62 Three-dimensional gain calculating unit-   63 Two-dimensional gain calculating unit-   64 Multiplier-   65 Multiplier-   91 Selecting unit-   92-1 through 92-4, and 92 Three-dimensional gain calculating unit-   93 Adder

The invention claimed is:
 1. A sound processing apparatus comprising: aspeaker selecting unit configured to select four processing-targetspeakers; a virtual speaker position determining unit configured todetermine a position of a virtual speaker based on the processing-targetspeakers selected by the speaker selecting unit; and a gain calculatingunit configured to calculate gains of the selected processing-targetspeakers based on a target sound image position and the determinedposition of the virtual speaker by performing three-dimensional vectorbase amplitude panning (VBAP).
 2. The sound processing apparatusaccording to claim 1, wherein the gains of the selectedprocessing-target speakers each have a value other than
 0. 3. A methodcomprising: using at least one processor to perform: selecting fourprocessing-target speakers; determining a position of a virtual speakerbased on the selected four processing-target speakers; and calculatinggains of the selected processing-target speakers based on a target soundimage position and the determined position of the virtual speaker byperforming three-dimensional vector base amplitude panning (VBAP). 4.The method of claim 3, wherein calculating the gains of the selectedprocessing-target speakers comprises calculating a gain of each of theselected processing-target speakers as a value other than
 0. 5. At leastone non-transitory computer-readable storage medium storing instructionsthat, when executed by at least one processor, cause the at least oneprocessor to perform a method comprising: selecting fourprocessing-target speakers; determining a position of a virtual speakerbased on the selected four processing-target speakers; and calculatinggains of the selected processing-target speakers based on a target soundimage position and the determined position of the virtual speaker byperforming three-dimensional vector base amplitude panning (VBAP). 6.The method of claim 5, wherein calculating the gains of the selectedprocessing-target speakers comprises calculating a gain of each of theselected processing-target speakers as a value other than 0.